Image processing apparatus and image processing method

ABSTRACT

The image processing apparatus according to the present invention includes a pixel extraction section that extracts an extraction region made up of a target pixel and a plurality of pixels included in a region peripheral to the target pixel, a similitude calculation section that calculates similitude of each pixel value in the peripheral region with respect to a pixel value of the target pixel, a similitude filter processing section that performs filter processing on the similitude, a first filter determining section that determines a first filter used in the filter processing of the similitude filter processing section, a second filter determining section that determines a second filter used for the extraction region based on the similitude subjected to the filter processing by the similitude filter processing section, and a noise reduction processing section that performs noise reduction processing on the target pixel based on the second filter determined by the second filter determining section.

This application claims benefit of Japanese Patent Application No.2009-021813 filed in Japan on Feb. 2, 2009, the entire contents of whichare incorporated herein by this reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus and animage processing method, and more particularly, to an image processingapparatus and an image processing method that perform noise reductionprocessing on an image signal.

2. Description of Related Art

Generally, an image pickup apparatus provided with an image pickupdevice such as a CCD or CMOS photoelectrically converts an imageoptically formed on an image pickup plane via lenses in micro regionspixel by pixel and thereby outputs an image signal as an electricsignal. Furthermore, the aforementioned image signal is amplified by anamplifier to a predetermined brightness level, digitized by an A/Dconverter and subjected to further processing as a digital image.

Various types of noise ascribable to the image pickup device are mixedinto the digital image digitized as described above. Examples of theabove described noise include dark current, fixed pattern noise causedby a variation in gain of the amplifier accompanying a variation of eachpixel and random noise as shot noise caused by statistical nature at thetime of photoelectric conversion. Furthermore, the statisticalcharacteristic of the aforementioned noise is known to vary with thebrightness level. Shot noise in particular has average amplitudeproportional to the square root of the brightness level.

On the other hand, for example, processing by a spatial smoothing filterusing spatial correlativity of the image signal and spatialnon-correlativity of noise is widely known as processing that reducesthe noise level of a digital image into which noise originating in theimage pickup device is mixed and can thereby improve an SN ratio.However, the spatial smoothing filter often has an adverse effect on anedge section of the image. Therefore, Japanese Patent ApplicationLaid-Open Publication No. 2006-302023 describes a noise reduction methodusing such an edge-preserving filter that a weighting factor of asmoothing filter adaptively varies between an edge section and partsother than the edge section of the image.

The noise reduction filter according to Japanese Patent ApplicationLaid-Open Publication No. 2006-302023 is a filter that adaptivelychanges weights according to a difference in pixel values between atarget pixel to be processed within a target region and a pixelperipheral thereto. To be more specific, the noise reduction filteraccording to Japanese Patent Application Laid-Open Publication No.2006-302023 reduces the weight to 0 when the absolute value of theaforementioned difference value is above a threshold SH, and isconfigured on the other hand, when the absolute value of theaforementioned difference value is equal to or below the threshold SH,using a filter coefficient whereby the absolute value of the differencevalue is converted to a weight using a function that monotonouslydecreases down to a predetermined value as the absolute value of theaforementioned difference value increases from 0 and monotonouslyincreases when the absolute value of the aforementioned difference valuereaches or exceeds the predetermined value. Using such a filter designedbased on the difference value between the target pixel and the pixelperipheral thereto makes it possible to suppress noise in a flat partwhile preserving the edge section of the image.

Furthermore, as the adaptive smoothing filter, the weight of whichadaptively varies according to the structure of an image, there are, forexample, filters called a “rational filter” and a “bilateral filter”expressed by following equation (1).

F(x,y)=N·exp{−(|spatial distance(x,y)/σ_(s))²/2}·exp{−(|pixel valuedifference(x,y)/σ_(d))²/2}  (1)

As shown in equation (1) above, the bilateral filter has a filtercoefficient obtained by multiplying a Gaussian filter corresponding to aspatial distance by a Gaussian filter corresponding to a pixel valuedifference. The first term of the right side of equation (1) aboveindicates that a fixed weight independent of the structure of an imageis given and the second term of the right side in equation (1) aboveindicates that a weight that adaptively varies depending on thestructure of the image is given.

The bilateral filter expressed by equation (1) above operates so as toexclude pixels having a large difference in pixel values between thepixel to be processed and a pixel peripheral thereto from smoothingprocessing as much as possible, and therefore never blunts edge sectionshaving large differences in pixel value. As a result, the bilateralfilter expressed by equation (1) above can reduce a large amount ofnoise while maintaining resolution, and can thereby obtain an effectivenoise reduction result.

The second term of the right side of equation (1) above indicates thatthe difference in pixel values used for smoothing is a difference on theorder of σ_(d) at most. When the value of this σ_(d) is fixed, aselection criterion of pixels used for smoothing becomes constant withinthe image. However, the amount of noise originating in the image pickupdevice varies depending on the brightness level as described above.Therefore, fixing value of σ_(d) is inconvenient when the processing isactually performed. In consideration of such circumstances, a techniqueof changing the value of σ_(d) of the second term of the right sideaccording to the amount of noise generated in the bilateral filterexpressed as equation (1) above is described in U.S. Patent ApplicationPublication No. 2007/0009175.

U.S. Patent Application Publication No. 2007/0009175 describes atechnique of storing a standard deviation value, which is an amount ofnoise determined according to the brightness level and measuredbeforehand as a table, outputting an amount of estimated noise withrespect to the brightness level of the pixel to be processed and causingthe value of σ_(d) to correspond to the amount of estimated noise.According to such a technique described in U.S. Patent ApplicationPublication No. 2007/0009175, it is possible to design a filter weightadaptable not only to a spatial structure of an image but also to thebrightness level, and thereby obtain an effective noise reduction resultwithout crushing even a smaller edge structure.

On the other hand, a rational filter expressed as equation (2) belowdoes not crush the micro structure so much as the aforementionedbilateral filter.

F(x,y)=T/(|pixel value difference|−T)  (2)

In the case of the rational filter expressed as equation (2) above, T isthe parameter that corresponds to σ_(d) above.

SUMMARY OF THE INVENTION

The image processing apparatus according to the present inventionincludes a pixel extraction section that extracts an extraction regionmade up of a target pixel and a plurality of pixels included in a regionperipheral to the target pixel, a similitude calculation section thatcalculates similitude of each pixel value in the peripheral region withrespect to a pixel value of the target pixel, a similitude filterprocessing section that performs filter processing on the similitude, afirst filter determining section that determines a first filter used inthe filter processing of the similitude filter processing section, asecond filter determining section that determines a second filter usedfor the extraction region based on the similitude subjected to thefilter processing by the similitude filter processing section, and anoise reduction processing section that performs noise reductionprocessing on the target pixel based on the second filter determined bythe second filter determining section.

The image processing method according to the present invention includesa pixel extracting step of extracting an extraction region made up of atarget pixel and a plurality of pixels included in a region peripheralto the target pixel, a similitude calculating step of calculatingsimilitude of each pixel value in the peripheral region with respect toa pixel value of the target pixel, a similitude filter processing stepof performing filter processing on the similitude, a first filterdetermining step of determining a first filter used for the filterprocessing in the similitude filter processing step, a second filterdetermining step of determining a second filter used for the extractionregion based on the similitude subjected to the filter processing in thesimilitude filter processing step, and a noise reduction processing stepof performing noise reduction processing on the target pixel based onthe second filter determined in the second filter determining step.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a function block diagram illustrating a configuration of mainparts of an image processing apparatus according to an embodiment of thepresent invention;

FIG. 2 is a function block diagram illustrating an example of a specificconfiguration of the noise reduction section in the image processingapparatus in FIG. 1;

FIG. 3 is a diagram illustrating an example of a frequencycharacteristic of a bilateral filter;

FIG. 4 is a diagram illustrating an example of a relationship between abrightness value and an amount of noise superimposed on a pixel;

FIG. 5 is a diagram illustrating an example of a relationship between aselection threshold of a filter type for processing difference valuesbetween pixel values and an amount of noise;

FIG. 6A is a schematic view illustrating an example of a frequencycharacteristic before and after filter processing of the differencevalues between pixel values;

FIG. 6B is a schematic view illustrating an example different from FIG.6A of a frequency characteristic before and after filter processing ofthe difference values between pixel values;

FIG. 7A is a diagram schematically illustrating filter processing on thedifference values between pixel values in an extraction region;

FIG. 7B is a diagram illustrating an example of filter coefficients whenthe extraction region is processed;

FIG. 7C is a diagram illustrating an example different from FIG. 7B offilter coefficients when the extraction region is processed;

FIG. 8 is a function block diagram illustrating an example of a specificconfiguration of the noise reduction processing section;

FIG. 9A is a diagram schematically illustrating the difference valuesbetween pixel values when high frequency noise is superimposed on a flatregion of 5×5 pixels;

FIG. 9B is a diagram schematically illustrating filter coefficientscalculated using the difference values between pixel values in thefilter used for the region in FIG. 9A;

FIG. 9C is a diagram schematically illustrating filter coefficientsobtained by operating a low pass filter fbr cutting high frequency noiseincluded in the difference values between pixel values in the filterused for the region in FIG. 9A;

FIG. 10A is a diagram schematically illustrating the difference valuesbetween pixel values when high frequency noise is superimposed on a flatregion of 5×5 pixels;

FIG. 10B is a diagram schematically illustrating filter coefficientscalculated using the difference values between pixel values in thefilter used for the region in FIG. 10A;

FIG. 10C is a diagram schematically illustrating filter coefficientsobtained by operating a low pass filter for cutting high frequency noiseincluded in the difference values between pixel values in the filterused for the region in FIG. 10A;

FIG. 11 is a diagram illustrating a relationship between two types ofcoefficient values to normalize the difference values between pixelvalues of a bilateral filter;

FIG. 12 is a function block diagram illustrating an example of aspecific configuration of a difference extraction region filterprocessing section according to a second embodiment of the presentinvention;

FIG. 13A is a diagram illustrating an example of filter coefficientsused in the difference extraction region filter processing section inFIG. 12;

FIG. 13B is a diagram illustrating an example different from FIG. 13A offilter coefficients used in the difference extraction region filterprocessing section in FIG. 12;

FIG. 13C is a diagram illustrating an example different from FIG. 13Aand FIG. 13B of filter coefficients used in the difference extractionregion filter processing section in FIG. 12;

FIG. 13D is a diagram illustrating an example different from FIG. 13A,FIG. 13B and FIG. 13C of filter coefficients used in the differenceextraction region filter processing section in FIG. 12;

FIG. 13E is a diagram illustrating an example different from FIG. 13A,FIG. 13B, FIG. 13C and FIG. 13D of filter coefficients used in thedifference extraction region filter processing section in FIG. 12; and

FIG. 14 is a flowchart illustrating a processing procedure for noisereduction processing in the configuration provided with the differenceextraction region filter processing section in FIG. 12.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Hereinafter, embodiments of the present invention will be described withreference to the accompanying drawings.

First Embodiment

FIG. 1 to FIG. 11 are related to a first embodiment of the presentinvention. FIG. 1 is a function block diagram illustrating aconfiguration of main parts of an image processing apparatus accordingto an embodiment of the present invention. FIG. 2 is a function blockdiagram illustrating an example of a specific configuration of the noisereduction section of the image processing apparatus in FIG. 1. FIG. 3 isa diagram illustrating an example of a frequency characteristic of abilateral filter. FIG. 4 is a diagram illustrating an example of arelationship between a brightness value and an amount of noisesuperimposed on a pixel. FIG. 5 is a diagram illustrating an example ofa relationship between a selection threshold of a filter type forprocessing difference values between pixel values and an amount ofnoise. FIG. 6A is a schematic view illustrating an example of afrequency characteristic before and after filter processing of thedifference values between pixel values. FIG. 6B is a schematic viewillustrating an example different from FIG. 6A of a frequencycharacteristic before and after filter processing of the differencevalues between pixel values. FIG. 7A is a diagram schematicallyillustrating filter processing on the difference values between pixelvalues in an extraction region. FIG. 7B is a diagram illustrating anexample of filter coefficients when the extraction region is processed.FIG. 7C is a diagram illustrating an example different from FIG. 7B offilter coefficients when the extraction region is processed.

FIG. 8 is a function block diagram illustrating an example of a specificconfiguration of the noise reduction processing section. FIG. 9A is adiagram schematically illustrating the difference values between pixelvalues when high frequency noise is superimposed on a flat region of 5×5pixels. FIG. 9B is a diagram schematically illustrating filtercoefficients calculated using the difference values between pixel valuesin the filter used for the region in FIG. 9A. FIG. 9C is a diagramschematically illustrating filter coefficients obtained by operating alow pass filter for cutting high frequency noise included in thedifference values between pixel values in the filter used for the regionin FIG. 9A. FIG. 10A is a diagram schematically illustrating thedifference values between pixel values when high frequency noise issuperimposed on a flat region of 5×5 pixels. FIG. 10B is a diagramschematically illustrating filter coefficients calculated using thedifference values between pixel values in the filter used for the regionin FIG. 10A. FIG. 10C is a diagram schematically illustrating filtercoefficients obtained by operating a low pass filter for cutting highfrequency noise included in the difference values between pixel valuesin the filter used for the region in FIG. 10A. FIG. 11 is a diagramillustrating a relationship between two types of coefficient values tonormalize the difference values between pixel values of a bilateralfilter.

As shown in FIG. 1, an image processing apparatus 110 is configured byincluding an image pickup section 100, a color signal separation section101, a plurality of noise reduction sections 102, an interpolationprocessing section 103, a color correction processing section 104, agradation correction section 105, a compression recording section 106and a system control section 107.

The image pickup section 100 is configured by including a lens and animage pickup device (not shown). An image formed on the image pickupdevice via the lens of the image pickup section 100 is photoelectricallyconverted and outputted as an analog signal. The analog signal is thenamplified by an amplification section (not shown), converted to digitaldata by an A/D conversion section (not shown) and then outputted to thecolor signal separation section 101.

In the present embodiment, the image pickup device of the image pickupsection 100 will be described on the premise of a Bayer array of singledevice with RGB primary color filters arranged in a checkered pattern ona chip of CCD or CMOS sensor or the like, one color arranged on eachpixel, but the image pickup device is not limited to such an array. Tobe more specific, the image pickup device of the image pickup section100 may have a multi configuration including a plurality of monochromeimage pickup devices and a dichroic prism and forming images of lightrays color-separated via the dichroic prism on the respective imagepickup devices. In this case, a signal separated by the color signalseparation section 101, which will be presented below, is different andthe interpolation processing section 103 becomes unnecessary.Furthermore, the image pickup device of the image pickup section 100 mayhave a configuration adaptable to multibands by combining theaforementioned two types of configuration.

The color signal separation section 101 separates digital data from theimage pickup section 100 into portions for R, Gr, Gb and B pixelsignals. The color signal separation section 101 then applies whitebalance processing to the respective separated pixel signals, multipliesthe R signal and B signal by the gain for the G signal and then outputsthe four color signals to their corresponding noise reduction sections102.

The noise reduction section 102 applies noise reduction processing toeach color signal outputted from the color signal separation section 101and then outputs each color signal after the noise reduction processingto the interpolation processing section 103.

For a color signal having only one color per pixel, the interpolationprocessing section 103 uses pixel values of the same color and adifferent color in pixels peripheral to the one pixel to createinterpolation signals of two colors, which do not exist in the onepixel. Through such processing, the interpolation processing section 103generates a synchronized RGB signal having three color signals per pixeland outputs the RGB signal to the color correction processing section104.

The color correction processing section 104 performs processing ofconverting the RGB signal in a device color space ascribable tocharacteristics of the image pickup section 100 as the output signalfrom the interpolation processing section 103 to a color region (e.g.,sRGB) of a monitor or the like to which the signal is outputted. Thecolor correction processing section 104 then outputs the RGB signalsubjected to the processing to the gradation correction section 105.

The gradation correction section 105 applies gradation conversionprocessing corresponding to the characteristics of the monitor or thelike, to which the signal is outputted, to the RUB signal from the colorcorrection processing section 104 and outputs the RGB signal after thegradation conversion processing to the compression recording section106.

The compression recording section 106 converts the RGB signal from thegradation correction section 105 to a YCbCr signal which is a brightnesscolor difference signal, applies high efficiency compression processingsuch as JPEG or MPEG to the YCbCr signal and then records the signalafter the high efficiency compression processing in a recording medium.

The system control section 107 outputs control signals for allowing theaforementioned respective sections to functionally operate processing inthe image processing apparatus 110 as a whole. That is, the respectivesections of the image processing apparatus 110 operate based on controlsignals outputted from the system control section 107 respectively.

Here, a specific configuration of the noise reduction section 102 willbe described.

As shown in FIG. 2, the noise reduction section 102 is configured byincluding a memory 200, a region extraction section 201, a differencevalue calculation section 202, a statistical amount calculation section20J, a noise amount estimation section 204, a filter coefficient table205, a filter coefficient selection section 206, a difference extractionregion filter processing section 207, a weighting factor conversion TUT208, a filter coefficient calculation section 209, a noise reductionprocessing section 210 and a control section 211.

The memory 200 receives one type of color signal outputted from thecolor signal separation section 101 as image data. The image data istemporarily stored in the memory 200 to absorb a time delay of noisereduction processing.

The region extraction section 201 reads the image data stored in thememory 200 and extracts a target pixel P(x0,y0) on which noise reductionprocessing is performed and a region image of, for example, 7×7 pixelsas a region of a predetermined size including the target pixel P in thecenter. The region extraction section 201 then outputs the extractedregion image to the difference value calculation section 202 and thenoise amount estimation section 204.

The difference value calculation section 202 calculates a differencevalue D(x, y) indicating similitude between the target pixel P(x0, y0)located in the center of the region image and a pixel P(x, y) in aregion peripheral thereto based on the region image from the regionextraction section 201 according to following equation (3).

D(x,y)=P(x,y)−P(x0,y0)  (3)

Here, suppose D(x0, y0)=0 in equation (3) above. Furthermore,hereinafter, the present embodiment will continue following explanationsassuming similitude as the difference value D(x, y), but the presentinvention need not be limited thereto. For example, the similitude maybe the absolute value of the difference value D(x, y) or the squarevalue of the difference value D(x, y).

The difference value calculation section 202 outputs the calculateddifference value D(x, y) to the statistical amount calculation section203 and the noise amount estimation section 204 respectively.

The statistical amount calculation section 203 calculates standarddeviation σ_(D) corresponding to the difference value D(x, y) from thedifference value calculation section 202 and outputs the standarddeviation σ_(D) to the filter coefficient selection section 206. Thestatistical amount calculation section 203 need not necessarilycalculate the standard deviation σ_(D) as the statistical amount but mayalso calculate, for example, a dynamic range of the difference valueD(x, y) in the extraction region.

On the other hand, the noise amount estimation section 204 calculatesthe extraction region of the difference value D(x, y) from thedifference value calculation section 202 and one representativebrightness value R corresponding to the target pixel P(x0, y0) of theextraction region. The representative brightness value R may becalculated, for example, as D(x0, y0)=P(x0, y0) or calculated as anaverage value in the extraction region of the difference value D(x, y)added to P(x0, y0) or calculated as a processing result of the bilateralfilter expressed as following equations (4), (5) and (6).

R=Σ _(x)Σ_(y)exp{−S(x,y)²/(2σ_(s) ²)}exp{−(D(x,y)²/(2σ_(d)²)}}·{D(x,y)+P(x0,y0)}/No  (4)

No=Σ_(x)Σ_(y)exp{−S(x,y)²/(2σ²)}exp{−(D(x,y)²/(2σ_(d) ²)}}  (5)

S(x,y)²=(x−x0)²+(y−y0)²  (6)

Here, suppose Σ_(x) and Σ_(y) in equations (4) and (5) above indicatethe sum totals within a range of values that x and y can take.

The noise amount estimation section 204 converts the representativebrightness value R to a noise estimate N(R) by applying therepresentative brightness value R to the noise model Nm(R) measuredbeforehand as shown, for example, in FIG. 4.

The noise model Nm(R) in the image pickup apparatus is calculated as afunction obtained by applying a least squares method to a polynomialapproximate curve obtained by taking images of a plurality of patcheswhich become a uniform brightness region beforehand using a specificimage pickup apparatus and then measuring an average brightness valueand standard deviation of a predetermined region in each patch. Such anoise model Nm(R) may retain the polynomial coefficient or may beconverted to a LUT as line graph data made up of a plurality of points.

The noise estimate N(R) is calculated, for example, by multiplying theaforementioned noise model Nm(R) by a function P of parameters (amountof gain, exposure time, temperature, . . . ) that causes the amount ofnoise to increase such as an amount of gain. ISO sensitivity, exposuretime, temperature of the sensor itself specified by the image pickupsection 100 and amount of gain accompanying white balance processing ofthe color signal separation section 101.

The noise amount estimation section 204 calculates the function P basedon the aforementioned respective parameters inputted via the controlsection 211. Suppose the function P is expressed so as to calculate afunction value such as a polynomial or LUT.

Therefore, the noise estimate N(R) is calculated by following equation(7).

N(R)=Nm(R)×P(amount of gain, exposure time, temperature, . . . )  (7)

The filter coefficient selection section 206 receives the standarddeviation σ_(D) from the statistical amount calculation section 203 andthe noise estimate N(R) from the noise amount estimation section 204.The filter coefficient selection section 206 determines a thresholdTh(R) based on the noise estimate N(R) and selects one filter from amonga plurality of filters stored in the filter coefficient table 205 basedon a judgment in magnitude between σ_(D) and Th(R).

As shown in equation (8) below, the threshold Th(R) is calculated as anadditional value of the noise estimate N(R) and a predetermined valueα(R).

Th(R)=N(R)+α(R)  (8)

As shown in FIG. 5, the threshold Th(R) is set so as to increase itsvalue as the noise estimate N(R) increases with an increase of the noiseestimate N(R) in accordance with the brightness value.

Upon detecting σ_(D)≧Th(R), the filter coefficient selection section 206selects a first filter corresponding to a structure region with strongedge strength. On the other hand, upon detecting σ_(D)<Th(R), the filtercoefficient selection section 206 selects a second filter correspondingto the region of a micro structure or a flat section. Furthermore, thefilter coefficient selection section 206 outputs the coefficient valueof the selected filter to the difference extraction region filterprocessing section 207 and outputs filter selection information SWindicating which of the first filter or second filter is selected to thefilter coefficient calculation section 209.

Here, a comparison/judgment between the threshold Th(R) and the standarddeviation σ_(D) of the extraction region of the difference value D(x, y)means a judgment as to whether or not there is a large edge componentwith respect to the noise estimate N(R) in the extraction region to beprocessed. That is, the filter coefficient selection section 206selectively changes the type of filter operating on the difference valueD(x, y) of the extraction region depending on the presence/absence of alarge edge.

The filter coefficient table 205 stores filters to be selected by thefilter coefficient selection section 206. For example, coefficients inFIG. 7B correspond to the aforementioned first filter and coefficientsin FIG. 7C correspond to the aforementioned second filter.

The aforementioned first filter includes a filter characteristic thatallows a signal to pass through all bands. On the other hand, theaforementioned second filter has a characteristic equivalent to that ofa low pass filter, and cuts high frequency noise together with astructure of a spatially high frequency component.

That is, when it is judged, as a result of the comparison/judgmentbetween the threshold Th(R) and the standard deviation σ_(D) of theextraction region of the difference value D(x, y), that there is a largeedge, the difference value D(x, y) of the extraction region is retainedas is, and the spatial frequency characteristic as shown, for example,in FIG. 6A is thereby kept and the edge component included in thedifference value D(x, y) of the extraction region is saved. On the otherhand, when it is judged as a result of the comparison/judgment betweenthe threshold Th(R) and the standard deviation σ_(D) of the extractionregion of the difference value D(x, y), that there is no large edge, asecond filter having a characteristic equivalent to that of the low passfilter is made to operate to change the spatial frequency characteristicas shown, for example, in FIG. 6B.

The filter characteristics of the first filter and the second filterstored in the filter coefficient table 205 are not necessarily based onthe filter coefficients shown in FIG. 7B and FIG. 7C. To be morespecific, the first filter may be configured as a steep low pass filterthat attenuates a frequency near the Nyquist frequency while retainingits frequency band as much as possible. In such a configuration, sincethe noise component near the Nyquist frequency can also be removed usingthe first filter, the influences of noise on the structure can bereduced. However, in the case of the aforementioned configuration, thefilter size of the first filter needs to be set to a size greater thanthe extraction region of the difference value D(x, y).

The difference extraction region filter processing section 207 appliesfilter processing to the difference value D(x, y) of the extractionregion based on the difference value D(x, y) of the extraction regionoutputted from the difference value calculation section 202 and thefilter coefficient outputted from the filter coefficient selectionsection 206 and then outputs the processing result to the filtercoefficient calculation section 209.

FIG. 7A is a schematic view corresponding to the filter processing onthe difference value D(x, y)=D_(xy) of the extraction region. By causinga two-dimensional filter with 3×3 taps shown in FIG. 7B or FIG. 7C toperform convolution processing per pixel on the difference value D(x, y)of the extraction region of 7×7 pixels shown in FIG. 7A, D′(x, y) isobtained as the processing result by the difference extraction regionfilter processing section 207, The region of D′(x, y) is a region madeup of 5×5 pixels around the position of the pixel to be processed ofD₃₃, obtained by removing a marginal area of one pixel in width thatcannot be subjected to filter processing from the extraction region of7×7 pixels shown, for example, in FIG. 7A.

The filter coefficient calculation section 209 calculates a smoothingprocessing filter coefficient with respect to the extraction regionextracted by the region extraction section 201 based on D′(x, y)outputted from the difference extraction region filter processingsection 207 and the filter selection information SW outputted from thefilter coefficient selection section 206.

The filter coefficient calculated by the filter coefficient calculationsection 209 corresponds to the bilateral filter expressed as followingequations (9) and (10) and is provided with 5×5 taps corresponding tothe size of D′(x, y).

F(x,y)=exp{−S(x,y)²/(2σ_(s) ²)}exp{−(D′(x,y)²/(2σ_(d)(SW)²)}  (9)

S(x,y)²=(x−x0)²+(y−y0)²  (10)

The Gaussian function exp{−S(x, y)²/(2σ_(s) ²)} of the first term of theright side in equation (9) above is a weighting factor based on thespatial distance and can be stored as a table by being calculatedbeforehand.

On the other hand, the weighting factor conversion Lurr 208 stores aconversion table for converting the filter selection information SW to aGaussian function value corresponding to the Gaussian functionexp{−(D′(x, y)²/(2σ_(d)(SW)²)} of the second term of the right side inequation (9) above as a plurality of lookup tables LUT_(G) beforehand.Such a configuration allows the aforementioned Gaussian function valueto be obtained by referring to a value of a storage location in thelookup table LUT_(G) (SW) where the address is D′(x, y).

The filter coefficient calculation section 209 outputs a filtercoefficient F(x, y) calculated using the lookup table LUT_(G) stored inthe weighting factor conversion LUT 208 and equations (9) and (10) aboveto the noise reduction processing section 210. Suppose normalizationprocessing on the filter coefficient F(x, y) calculated by the filtercoefficient calculation section 209 will be performed after product-sumoperation processing by the noise reduction processing section 210.

Of the extraction region of 7×7 pixels outputted from the regionextraction section 201, the noise reduction processing section 210selects the target pixel P(x0, y0) located in the center of theextraction region and 5×5 pixels peripheral to the target pixel P(x0,y0). After that, the noise reduction processing section 210 performsproduct-sum operation processing between 5×5 pixels peripheral to thetarget pixel P(x0, y0) and the filter coefficient of 5×5 taps outputtedfrom the filter coefficient calculation section 209, further calculatesthe sum total No of the filter coefficients to perform normalizationprocessing and thereby obtains a pixel to be subjected to smoothingprocessing P′(x0, y0). The noise reduction processing section 210 thenoutputs the pixel to be subjected to smoothing processing P′(x0, y0) asthe processing result as a noise reduction processing output pixel valuePo(x0, y0).

Assuming the noise reduction processing section 210 performs processingsubstantially the same as the aforementioned processing, the noisereduction processing section 210 may have a configuration as shown, forexample, in FIG. 8.

The noise reduction processing section 210 is configured by including aproduct-sum operation processing section 801, a normalization processingsection 802, a noise estimate calculation section 803 and a coringprocessing section 804 as shown in FIG. 8.

The product-sum operation processing section 801 performs product-sumoperation processing among the target pixel P(x0, y0) included in theextraction region, P(x, y) made up of 5×5 pixels peripheral to thetarget pixel P(x0, y0) and filter coefficient F(x, y) with 5×5 taps, andthen outputs the product-sum operation processing result P″(x0, y0) tothe normalization processing section 802.

The normalization processing section 802 calculates a sum total of thefilter coefficient F(x, y); No(SW)Σ_(x)Σ_(y)exp{−S(x, y)²/(2σ_(s)²)}exp{−(D′(x, y)²/(2σ_(d)(SW)²)}, converts the calculation result to areciprocal according to the reciprocal conversion table of No(SW) storedin itself, then multiplies the converted value by the product-sumoperation processing result P″(x0, y0) and thereby calculates the pixelto be subjected to smoothing processing P′(x0, y0). The normalizationprocessing section 802 then outputs the calculated pixel to be subjectedto smoothing processing P′(x0, y0) to the noise estimate calculationsection 803 and the coring processing section 804 respectively.

The noise estimate calculation section 803 calculates a noise estimateN(R) as R=P′(x0, y0) based on the noise model Nm(R) shown in FIG. 4 andimage taking condition information (exposure time, amount of gain. ISOsensitivity and sensor temperature or the like) outputted from thecontrol section 211. Here, the noise estimate in the noise estimatecalculation section 803 is calculated using the same method as themethod described as the explanation of the noise amount estimationsection 204. The noise estimate calculation section 803 then outputs thecalculated noise estimate N(R) to the coring processing section 804.

The coring processing section 804 obtains a final output pixel valuePo(x0, y0) by performing coring processing based on the target pixelP(x0, y0) outputted from the region extraction section 201, the pixel tobe subjected to smoothing processing P′(x0, y0) outputted from thenormalization processing section 802 and the noise estimate N(P′(x0,y0)) outputted from the noise estimate calculation section 803.

To be more specific, in the case of |P(x0, y0)−P′(x0, y0)|<N(P′(x0,y0)), the coring processing section 804 regards P′(x0, y0) as the finaloutput pixel value Po(x0, y0). Furthermore, in the case of P(x0,y0)−P′(x0, y0)≧N(P′(x0, y0)), the coring processing section 804 regardsP(x0, y0)−N(P′(x0, y0)) as the final output pixel value Po(x0, y0). Onthe other hand, in the case of P(x0, y0)−P′(x0, y0)≦−N(P′(x0, y0)), thecoring processing section 804 regards P(x0, y0)+N(P′(x0, y0)) as thefinal output pixel value Po(x0, y0).

Here, an operational aspect of the configuration of the above describednoise reduction section 102 will be described.

The bilateral filter as the edge-preserving smoothing filter used forthe configuration of the noise reduction section 102 is a filter havinga frequency characteristic which differs depending on the structure ofthe image region applied as shown, for example, in FIG. 3.

Such a variation of the frequency characteristic is caused by a weightof the bilateral filter obtained by the brightness difference. That is,when the pixel where the value obtained by dividing the absolute valueof the brightness difference by σ_(d) (value of |brightnessditierence|/σ_(d)) is a value greater than 1 is located in theextraction region to be subjected to smoothing processing (regiondefined by the number of filter taps, and a region made up of 5×5 pixelsin size in the aforementioned explanation), the value of the Gaussianfunction takes a value close to 0, and therefore the pixel issubstantially excluded from the pixels used to perform smoothingprocessing.

With such nature, even when the pixel to be processed is located on anedge boundary, if it is an edge where the brightness difference from thepixel peripheral to the pixel to be processed is large, the pixellocated in the direction orthogonal to the edge direction does not jointhe smoothing processing, and it is thereby possible to obtain a filtercapable of saving a spatial frequency band of the edge section.

On the other hand, in the case of a micro structure region where abrightness difference between a pixel to be processed and a pixelperipheral to the pixel to be processed is small, that is, the valueobtained by dividing the absolute value of the brightness difference byσ_(d) (value of |brightness difference|/σ_(d)) is a value smaller than1, the value of the Gaussian function is a value close to 1, andtherefore all pixels including the micro structure region are used forsmoothing processing. In such a case, the frequency characteristic ofthe smoothing filter becomes closer to the low pass filter of thecharacteristic determined by another Gaussian function concerning thespatial distance of the bilateral filter.

Therefore, the magnitude of the parameter σ_(d) with respect to theabsolute value of the brightness difference of the bilateral filterbecomes a predominant element as to whether or not the micro structureof the image can be maintained. That is, setting σ_(d) as a relativelysmall value makes it possible to retain a micro structure correspondingto a region where the absolute value of the brightness difference isrelatively small. However, in such a case, when the absolute value ofthe brightness difference comes to have magnitude corresponding to theamount of noise (brightness difference obtained by noise beingsuperimposed), the variation by noise causes the value obtained bydividing the absolute value of the brightness difference by σ_(d) (valueof brightness difference/a) to change considerably, and thereby has alarge influence on the filter coefficient.

FIG. 9A to FIG. 9C, and FIG. 10A to FIG. 10C are drawings schematicallyillustrating the aforementioned explanation. Shading of each cell inFIG. 9A and FIG. 10A visually expresses a difference in the differencevalue (the darker the color, the larger the difference value).Furthermore, shading of each cell in FIG. 9B, FIG. 9C, FIG. 10B and FIG.10C visually expresses a difference in the filter coefficient (weight ofa blank cell is greater).

FIG. 9A schematically illustrates the difference value D(x, y) when highfrequency noise (amount of noise N) is superimposed on the flat regionof 5×5 pixels.

In such a condition, if N≈σ_(d) is assumed, the calculated filtercoefficient varies in accordance with the pattern of high frequencynoise as shown, tier example, in FIG. 9B as the variation of D(x,y)/σ_(d) increases and pixels to be used for smoothing are limited, andit is not possible to obtain a sufficient noise reduction effect as aconsequence.

When σ_(d) is reduced down to a value close to the noise level for thepurpose of maintaining the micro structure as described above, there isno more noise reduction effect. On the other hand, when σ_(d) isincreased, a sufficient noise reduction effect can be obtained, butnonetheless, the micro structure cannot be maintained.

To overcome this reciprocity, according to the noise reduction section102 of the present embodiment, when the target pixel to be subjected tonoise reduction processing and the region image peripheral to the targetpixel are judged to be a micro structure region, D′(x, y) is determinedby operating a low pass filter for cutting high frequency noise includedin the difference value D(x, y). This allows D′(x, y)/σ_(d) to be avalue smaller than 1 and makes it possible to obtain all filtercoefficients of 5×5 taps as uniform coefficients as shown, for example,in FIG. 9C.

On the other hand, FIG. 10A schematically illustrates the differencevalue D(x, y) when high frequency noise is superimposed on theextraction region provided with a micro structure (small edge) in thespatially longitudinal direction.

When a filter coefficient is calculated from the difference value D(x,y) corresponding to the micro structure region in FIG. 10A, since pixelsto be used for smoothing are limited as shown, for example, in FIG. 10B,this filter coefficient is a filter coefficient whereby any sufficientnoise reduction effect cannot be obtained. On the other hand, usingD′(x, y) operated by a low pass filter for cutting high frequency noiseincluded in the difference value D(x, y), it is possible to obtain afilter coefficient that can use many pixels existing at pixel positionsalong the low-frequency micro structure in the longitudinal directionfor smoothing as shown, for example, in FIG. 10C.

That is, even if σ_(d) is reduced, it is possible to obtain acoefficient with influences of noise suppressed to a minimum bycalculating a difference value D′(x, y) whose frequency band in whichmany noise components relative to frequency components of the imagestructure are expected to be included is cut down and then calculating acoefficient of the bilateral filter.

When a frequency band to be emphasized as the structure of an image isknown beforehand, a band pass filter for extracting the frequency bandmay also be operated. In this case, it is also possible to obtain aneffective noise reduction result while saving the image structure.

Furthermore, when the number of taps of the low pass filter operating onthe difference value D(x, y) cannot be increased as in the case of FIG.7C, D′(x, y) may be multiplied a predetermined times to obtain theamplitude of the micro structure to be retained or σ_(d) when the lowpass filter is not operated may be reduced so as to approximate σ_(dL)when the low pass filter is operated.

FIG. 11 is a diagram illustrating a relationship between σ_(d) andσ_(dL) and illustrates an example of a case where σ_(d) varies dependingon the amount of noise N(R). In this example, σ_(dL) (n(R)) during lowpass filter processing corresponding to the aforementioned second filtermay be set to σ_(dL)(n(R))=βσ_(d)(n(R)) (where β is a predeterminedconstant equal to or below 1). This makes it possible to calculate acoefficient of a more suitable bilateral filter capable of excludingnoise influences in a region relatively susceptible to noise such as amicro structure region.

On the other hand, when the region is judged to be a structure regionhaving sufficient edge strength, the coefficient of the bilateral filteris not relatively susceptible to noise. Therefore, the noise reductionsection 102 of the present embodiment calculates a filter coefficientwithout applying band restrictions due to filter processing to thedifference value D(x, y). This prevents pixels corresponding to astructure with no correlation from being added to smoothing processing,and can thereby reduce noise while maintaining edge preservingperformance.

Furthermore, the aforementioned filter coefficient calculationprocessing is not limited to the bilateral filter and can be likewiseused for a filter that uses the difference value D(x, y) to calculate afilter coefficient such as a rational filter.

As described above, the image processing apparatus 110 of the presentembodiment calculates a filter coefficient of the bilateral filter usinga difference value D′(x, y) as a processing result of applyingprocessing using a low pass filter or band pass filter to the differencevalue D(x, y) according to the structure of an image region. Thus,according to the image processing apparatus 110 of the presentembodiment, it is possible to obtain a high noise reduction effect whilemaintaining the micro structure as much as possible.

Second Embodiment

FIG. 12 to FIG. 14 are related to a second embodiment of the presentinvention. FIG. 12 is a function block diagram illustrating an exampleof a specific configuration of a difference extraction region filterprocessing section according to the second embodiment of the presentinvention. FIG. 13A is a diagram illustrating an example of filtercoefficients used in the difference extraction region filter processingsection in FIG. 12. FIG. 13B is a diagram illustrating an exampledifferent from FIG. 13A of filter coefficients used in the differenceextraction region filter processing section in FIG. 12. FIG. 13C is adiagram illustrating an example different from FIG. 13A and FIG. 13B offilter coefficients used in the difference extraction region filterprocessing section in FIG. 12. FIG. 13D is a diagram illustrating anexample different from FIG. 13A, FIG. 138 and FIG. 13C of filtercoefficients used in the difference extraction region filter processingsection in FIG. 12. FIG. 13E is a diagram illustrating an exampledifferent from FIG. 13A, FIG. 13B, FIG. 13C and FIG. 13D of filtercoefficients used in the difference extraction region filter processingsection in FIG. 12, FIG. 14 is a flowchart illustrating a processingprocedure for noise reduction processing in the configuration providedwith the difference extraction region filter processing section in FIG.12.

In the following explanations, detailed descriptions of componentssimilar to those in the first embodiment will be omitted. Furthermore,the configuration of the noise reduction section of the presentembodiment has a configuration similar to that of the noise reductionsection 102 according to the first embodiment. Therefore, suppose partsdifferent from those of the noise reduction section 102 in the firstembodiment will be mainly described in the present embodiment. To bemore specific, processing contents of the filter coefficient selectionsection 206 and the difference extraction region filter processingsection 207 in the present embodiment are different from those of thefirst embodiment, and therefore these processing contents will be mainlydescribed.

As shown in FIG. 12, the difference extraction region filter processingsection 207 is configured by including a first filter processing section1201, a second filter processing section 1202, a third filter processingsection 1203, a D range (dynamic range) calculation comparison section1204 and a similitude selection section 1205.

Furthermore, according to the difference extraction region filterprocessing section 207 shown in FIG. 12, a difference value D(x, y)outputted from the difference value calculation section 202 is inputtedto the first filter processing section 1201, the second filterprocessing section 1202 and the third filter processing section 1203respectively.

On the other hand, the filter coefficient selection section 206 selectsa predetermined filter from among five types of filter coefficientgroups stored in the filter coefficient table 205 shown, for example, inFIG. 13A to FIG. 13E. The filter coefficient selection section 206 thenoutputs the selected predetermined filter to the first filter processingsection 1201, the second filter processing section 1202 and the thirdfilter processing section 1203 respectively.

The first filter processing section 1201, the second filter processingsection 1202 and the third filter processing section 1203 perform filterprocessing in parallel. Here, suppose the difference value D(x, y) meansdifference values (D₀₀ to D₆₆) of the 7×7-pixel region shown in FIG. 7A.

When the filter coefficient selection section 206 selects apredetermined filter from among the five filter coefficient groupsstored in the filter coefficient table 205, to be more specific, thefollowing processing is performed.

First, the filter coefficient selection section 206 makes a judgment asto whether a region is a structure region with strong edge strength or amicro structure or a flat section region using the above describedjudgment method in the first embodiment.

When the region is judged to be a structure region with strong edgestrength, the filter coefficient selection section 206 selects thefilter coefficients in FIG. 13A and outputs the filter coefficients tothe first filter processing section 1201, the second filter processingsection 1202 and the third filter processing section 1203 respectively.Therefore, in this case, the three filter processing sections 1201, 1202and 1203 output the same processing result. Therefore, for the purposeof omitting unnecessary processing, processing of, for example, thesecond filter processing section 1202 and the third filter processingsection 1203 may be stopped to reduce the output in this case to 0.

On the other hand, when the region is judged to be a micro structure ora flat section region, the filter coefficient selection section 206selects the filter coefficients in FIG. 13B to be outputted to the firstfilter processing section 1201, selects the filter coefficients in FIG.13D to be outputted to the second filter processing section 1202 andoutputs all the five types of filter coefficients in FIG. 13A, FIG. 13B,FIG. 13C, FIG. 13D and FIG. 13E to the third filter processing section1203.

In this case, the first filter processing section 1201 performssmoothing processing in the horizontal direction and outputs Dh′(x, y)as the processing result. Furthermore, the second filter processingsection 1202 performs smoothing processing in the vertical direction andoutputs Dv′(x, y) as the processing result. Furthermore, the thirdfilter processing section 1203 performs smoothing processing in a radialdirection with respect to the center position and outputs De(x, y) asthe processing result.

Since the first filter processing section 1201 and the second filterprocessing section 1202 multiply one type of filter coefficients withoutdepending on the filter processing position, the processing contents areclear without showing details thereof. On the other hand, as for thethird filter processing section 1203, the type of filter coefficientsneeds to be changed by identifying the filter processing position, andtherefore details of the processing contents will be shown below.

First, the filter in FIG. 13A is defined as F0(n, m), the filter in FIG.13B is defined as F1(n, m), the filter in FIG. 13C is defined as F2(n,m), the filter in FIG. 13D is defined as F3(n, m) and the filter in FIG.13E is defined as F4(n, m). In such a case. Dr′(x, y)D_(xy)′ as thefilter processing result of difference value D(x, y)=D_(xy) is expressedas equation (11) to equation (35) as follows.

D ₁₁′=Σ_(n)Σ_(m) F2(n,m)D _(nm)/4  (11)

D ₁₂′=Σ_(n)Σ_(m) {F2(n,m)+F3(n,m)}D _(n(m+1))/8  (12)

D ₁₃′=Σ_(n)Σ_(m) F3(n,m)D _(n(m+2))/4  (13)

D ₁₄′=Σ_(n)Σ_(m) {F3(n,m)+F4(n,m)}D _(n(m+3))/8  (14)

D ₁₅′=Σ_(n)Σ_(m) F4(n,m)D _(n(m+4))/4  (15)

D ₂₁′=Σ_(n)Σ_(m) {F1(n,m)+F2(n,m)}D _((n+1)m)/8  (16)

D ₂₂′=Σ_(n)Σ_(m) F2(n,m)D _((n+1)(m+1))/4  (17)

D ₂₃′=Σ_(n)Σ_(m) F3(n,m)D _((n+1)(m+2))/4  (18)

D ₂₄′=Σ_(n)Σ_(m) F4(n,m)D _((n+1)(m+3))/4  (19)

D ₂₅′=Σ_(n)Σ_(m) {F1(n,m)+F4(n,m)}D _((n+1)(m+4))/8  (20)

D ₃₁′=Σ_(n)Σ_(m) F1(n,m)D _((n+2)m)/4  (21)

D ₃₂′=Σ_(n)Σ_(m) F1(n,m)D _((n+2)(m+1))/4  (22)

D ₃₃′=Σ_(n)Σ_(m) F0(n,m)D _((n+2)(m+3))  (23)

D ₃₄′=Σ_(n)Σ_(m) F1(n,m)D _((n+2)(m+3))/4  (24)

D ₃₅′=Σ_(n)Σ_(m) F1(n,m)D _((n+2)(m+4))/4  (25)

D ₄₁′=Σ_(n)Σ_(m) {F1(n,m)+F4(n,m))}D _((n+3)m)/8  (26)

D ₄₂′=Σ_(n)Σ_(m) F4(n,m)D _((n+3)(m+1))/4  (27)

D ₄₃′=Σ_(n)Σ_(m) F3(n,m)D _((n+3)(m+2))/4  (28)

D ₄₄′=Σ_(n)Σ_(m) F2(n,m)D _((n+3)(m+3))/4  (29)

D ₄₅′=Σ_(n)Σ_(m) {F1(n,m)+F2(n,m)}D _((n+3)(m+4))/8  (30)

D ₅₁′=Σ_(n)Σ_(m) F4(n,m)D _((n+4)m)/4  (31)

D ₅₂′=Σ_(n)Σ_(m) {F3(n,m)+F4(n,m)}D _((n+4)(m+1))/8  (32)

D ₅₃′=Σ_(n)Σ_(m) F3(n,m)D _((n+4)(m+2))/4  (33)

D ₅₄′=Σ_(n)Σ_(m) {F2(n,m)+F3(n,m)}D _((n+4)(m+3))/8  (34)

D ₅₅′=Σ_(n)Σ_(m) F2(n,m)D _((n+4)(m+4))/4  (35)

Suppose n and m in equation (11) to equation (35) above take a value ofany one of 0, 1 and 2. Furthermore, suppose Σ_(n) and Σ_(m) in equation(11) to equation (35) indicate the sum totals in the range of valuesthat n and m can take.

Dh′(x, y) as the processing result in the first filter processingsection 1201, Dv′(x, y) as the processing result in the second filterprocessing section 1202 and Dr′(x, y) as the processing result in thethird filter processing section 1203 are outputted to the D rangecalculation comparison section 1204 and the similitude selection section1205 as a state in which the region of 5×5 pixels centered on x=y=3 isextracted for the 7×7-pixel region in FIG. 7A.

The D range calculation comparison section 1204 obtains maximum valuesMaxH, MaxV and MaxR and minimum values MinH, MinV and MinR correspondingto three types of 5×5 pixels; Dh′(x, y), Dv′(x, y) and Dr′(x, y)respectively. The D range calculation comparison section 1204 calculatesdynamic ranges corresponding to the three types of 5×5 pixels: Dh′(x,y), Dv′(x, y) and De(x, y) by calculating DRh=MaxH−MinH, DRv=MaxV−MinVand DRr=MaxR−MinR respectively. Furthermore, the D range calculationcomparison section 1204 compares three dynamic ranges of DRh, DRv andDRr, and thereby outputs filter information of the filter processingsection (one of first, second and third filter processing sections whosedynamic range becomes a maximum to the similitude selection section1205.

The similitude selection section 1205 selects the correspondingdifference values of one type of 5×5 pixels subjected to filterprocessing out of Dh′(x, y), Dv′(x, y) and Dr′(x, y) based on the filterinformation from the D range calculation comparison section 1204 andoutputs the selection result to the filter coefficient calculationsection 209.

The aforementioned processing adopts a configuration in which threetypes of direction-dependent filter processing are performed inparallel, but it is also possible to adopt a configuration that matchesthe two-dimensional micro structure of more patterns by furtherincreasing the number of the types thereof. To be more specific, byadding fourth filter processing that processes only F2(n, m) and fifthfilter processing that processes only F4(n, m) to the aforementionedthree types of filter processing, it is possible to adopt aconfiguration in which a total of five types of direction-dependentfilter processing are performed in parallel.

In the first embodiment, an isotropic low pass filter or a band passfilter is adopted as a filter operating on the difference value D(x, y),whereas in the present embodiment, smoothing processing results in aplurality of directions are acquired through processing using five typesof smoothing filters having directionality as shown, for example, inFIG. 13A to FIG. 13E and one smoothing processing result in thedirection in which the highest contrast is obtained (edge direction ofthe micro structure) is then selected from among smoothing processingresults in the plurality of directions. This makes it possible to obtainD′(x, y) in a state in which high frequency noise is reduced whilesaving the micro structure of the target pixel to a maximum extent, andthereby obtain filter coefficients of the bilateral filter that matchthe micro edge structure as a consequence.

Here, the processing procedure for the aforementioned noise reductionprocessing will be described with reference to a flowchart in FIG. 14.

First, a pixel region made up of a pixel to be processed P(x0, y0) and apixel P(x, y) of a predetermined region peripheral to the pixel to beprocessed (e.g., 7×7 pixels) is extracted from a single color imagestored in the memory (step 1401).

Next, a difference value D(x, y) is calculated between the pixel to beprocessed P(x0, y0) located in the center of the extracted 7×7-pixelregion and the pixel P(x, y) peripheral to the pixel to be processed(step 1402).

Next, a standard deviation σ_(D) is calculated for an extraction regionof 7×7 pixels of the difference value D(x, y) (step 1403).

The amount of noise included in the pixel to be processed P(x, y) isestimated and a threshold Th to be used to judge whether or not thedifference value D(x, y) of the extraction region is a micro structureregion is determined based on the estimated amount of noise (step 1404).

Here, the aforementioned amount of noise can be calculated from afunction (or line graph table) that stores the amount of noise (standarddeviation) corresponding to the brightness level obtained by measuringthe amount of noise generated in the image pickup section 100 as a noisemodel beforehand and image taking parameters (amount of gain, exposuretime and temperature or the like).

Next, a standard deviation σ_(D) of the calculated extraction region iscompared with a micro structure region judgment threshold Th (step1405).

When it is judged in the processing of step 1405 that the threshold This equal to or above the standard deviation σ_(D), the extraction regionof the difference value D(x, y) obtained through the processing in step1402 is subjected to filter processing by selecting a first filter thatretains the frequency band up to the high frequency component, and D′(x,y) is thereby calculated (step 1406).

On the other hand, when it is judged in the processing of step 1405 thatthe threshold Th is less than the standard deviation σ_(D), M types offilters are selected for the extraction region of the difference valueD(x, y) obtained through the processing in step 1402 and subjected tofilter processing respectively, and M types of D′(x, y) are therebycalculated (step 1407). M D ranges corresponding to the M types ofdifference value D′(x, y) are calculated respectively (step 1408) and adifference value D′(x, y) after the filter processing that correspondsto a maximum D range is then selected (step 1409).

After that, based on the difference value D′(x, y) selected in step1409, filter coefficients corresponding to the pixel to be processedP(x0, y0) to be subjected to noise reduction processing and apredetermined peripheral region (5×5 pixels) of the pixel to beprocessed are calculated (step 1410).

By applying filter processing to the pixel P(x0, y0) at the processingtarget position and the predetermined peripheral region (5×5 pixels) ofthe pixel to be processed based on the filter coefficients calculated instep 1410, an output pixel value Po(x0, y0) is obtained whichcorresponds to the pixel P(x0, y0) with reduced noise (step 1411).

When the process in step 1411 is completed, it is judged whether or notprocessing of all pixels to be processed has been completed (step 1412),and if not completed, a series of processes from step 1401 is repeatedlyperformed. When processing of all pixels has been completed, the seriesof noise reduction processes is finished.

Through the above described processing of the present embodiment, it ispossible to obtain a sufficient noise reduction effect while retainingthe micro structure which has been conventionally crushed in noisereduction carried out using a smoothing filter using a spatialcorrelation.

Particularly, the above described processing of the present embodimenteffectively operates on an image provided with a micro structure withoutsteep edges such as blood vessel, like a medical image, for example, anendoscope image whose object is limited beforehand. That is, accordingto the above described processing of the present embodiment, weights aregiven to an important intermediate band out of spatial frequencies and afilter coefficient is calculated using a parameter D′(x, with influencesof high frequency noise suppressed, and it is thereby possible to reducenoise while retaining micro structures such as blood vessels.

The present invention is not limited to the above described embodiments,and it goes without saying that various modifications and applicationsare possible without departing from the spirit and scope of the presentinvention.

1. An image processing apparatus comprising: a pixel extraction sectionthat extracts an extraction region made up of a target pixel and aplurality of pixels included in a region peripheral to the target pixel;a similitude calculation section that calculates similitude of eachpixel value in the peripheral region with respect to a pixel value ofthe target pixel; a similitude filter processing section that performsfilter processing on the similitude; a first filter determining sectionthat determines a first filter used in the filter processing of thesimilitude filter processing section; a second filter determiningsection that determines a second filter used for the extraction regionbased on the similitude subjected to the filter processing by thesimilitude filter processing section; and a noise reduction processingsection that performs noise reduction processing on the target pixelbased on the second filter determined by the second filter determiningsection.
 2. The image processing apparatus according to claim 1, whereinthe similitude is a difference value of each pixel value of theperipheral region with respect to the pixel value of the target pixel.3. The image processing apparatus according to claim 1, wherein thefirst filter determining section determines the first filter based on astatistical amount of similitude of the extraction region and an amountof estimated noise of the target pixel.
 4. The image processingapparatus according to claim 3, wherein the second filter determiningsection determines a characteristic parameter of the second filter usedfor the extraction region based on the first filter determined in thefirst filter determining section.
 5. The image processing apparatusaccording to claim 4, wherein the characteristic parameter of the secondfilter is an amount of normalization with respect to the similitude. 6.The image processing apparatus according to claim 1, wherein the secondfilter determining section determines the second filter so that thegreater the absolute value of similitude subjected to the filterprocessing, the smaller the weighting factor value becomes.
 7. The imageprocessing apparatus according to claim 6, wherein the second filterdetermined by the second filter determining section is a bilateralfilter.
 8. The image processing apparatus according to claim 3, whereinthe first filter determining section selects a plurality of filtercandidates as the first filter used for the filter processing, and theimage processing apparatus further comprises a similitude selectionsection that selects one type of similitude from among the plurality oftypes of similitude subjected to the filter processing by the pluralityof filter candidates.
 9. An image processing method comprising: a pixelextracting step of extracting an extraction region made up of a targetpixel and a plurality of pixels included in a region peripheral to thetarget pixel; a similitude calculating step of calculating similitude ofeach pixel value in the peripheral region with respect to a pixel valueof the target pixel; a similitude filter processing step of performingfilter processing on the similitude; a first filter determining step ofdetermining a first filter used for the filter processing in thesimilitude filter processing step; a second filter determining step ofdetermining a second filter used for the extraction region based on thesimilitude subjected to the filter processing in the similitude filterprocessing step; and a noise reduction processing step of performingnoise reduction processing on the target pixel based on the secondfilter determined in the second filter determining step.
 10. The imageprocessing method according to claim 9, wherein the similitude is adifference value of each pixel value of the peripheral region withrespect to the pixel value of the target pixel.
 11. The image processingmethod according to claim 9, wherein in the first filter determiningstep, the first filter is determined based on a statistical amount ofsimilitude of the extraction region and an amount of estimated noise ofthe target pixel.
 12. The image processing method according to claim 11,wherein in the second filter determining step, a characteristicparameter of the second filter used for the extraction region isdetermined based on the first filter determined in the first filterdetermining step.
 13. The image processing method according to claim 12,wherein the characteristic parameter of the second filter is an amountof normalization with respect to the similitude.
 14. The imageprocessing method according to claim 9, wherein in the second filterdetermining step, the second filter is determined so that the greaterthe absolute value of similitude subjected to the filter processing, thesmaller the weighting factor value becomes.
 15. The image processingmethod according to claim 14, wherein the second filter determined inthe second filter determining step is a bilateral filter.
 16. The imageprocessing method according to claim 11, wherein in the first filterdetermining step, a plurality of filter candidates are selected as thefirst filter used for the filter processing, and the image processingmethod further comprises a similitude selecting step of selecting onetype of similitude from among the plurality of types of similitudesubjected to the filter processing by the plurality of filtercandidates.